Adaptive Optimization of Very Large JoinQueries

نویسندگان

  • Thomas Neumann
  • Bernhard Radke
چکیده

The use of business intelligence tools and other means to generate queries has led to great variety in the size of join queries. While most queries are reasonably small, join queries with up to a hundred relations are not that exotic anymore, and the distribution of query sizes has an incredible long tail. The largest real-world query that we are aware of accesses more than 4,000 relations. This large spread makes query optimization very challenging. Join ordering is known to be NP-hard, which means that we cannot hope to solve such large problems exactly. On the other hand most queries are much smaller, and there is no reason to sacrifice optimality there. This paper introduces an adaptive optimization framework that is able to solve most common join queries exactly, while simultaneously scaling to queries with thousands of joins. A key component there is a novel search space linearization technique that leads to near-optimal execution plans for large classes of queries. In addition, we describe implementation techniques that are necessary to scale join ordering algorithms to these extremely large queries. Extensive experiments with over 10 different approaches show that the new adaptive approach proposed here performs excellent over a huge spectrum of query sizes, and produces optimal or near-optimal solutions for most common queries.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

A limited memory adaptive trust-region approach for large-scale unconstrained optimization

This study concerns with a trust-region-based method for solving unconstrained optimization problems. The approach takes the advantages of the compact limited memory BFGS updating formula together with an appropriate adaptive radius strategy. In our approach, the adaptive technique leads us to decrease the number of subproblems solving, while utilizing the structure of limited memory quasi-Newt...

متن کامل

Economic Load Dispatch using PSO Algorithm Based on Adaptive Learning Strategy Considering Valve point Effect

Abstract: In recent years due to problems such as population growth and as a result increase in demand for electrical energy, power systems have been faced with new challenges that not existed in the past. One of the most important issues in modern power systems is economic load dispatch, which is a complex optimization problem with a large number of variables and constraints. Due to the comple...

متن کامل

Market Adaptive Control Function Optimization in Continuous Cover Forest Management

Economically optimal management of a continuous cover forest is considered here. Initially, there is a large number of trees of different sizes and the forest may contain several species. We want to optimize the harvest decisions over time, using continuous cover forestry, which is denoted by CCF. We maximize our objective function, the expected present value, with consideration of stochastic p...

متن کامل

Application of adaptive sampling in fishery part 2: Truncated adaptive cluster sampling designs

There are some experiences that researcher come across quite number of time for very large networks in the initial samples such that they cannot finish the sampling procedure. Two solutions have been proposed and used by marine biologists which we discuss in this article: i) Adaptive cluster sampling based on order statistics with a stopping rule, ii) Restricted adaptive cluster sampling. Until...

متن کامل

An adaptive modified firefly algorithm to unit commitment problem for large-scale power systems

Unit commitment (UC) problem tries to schedule output power of generation units to meet the system demand for the next several hours at minimum cost. UC adds a time dimension to the economic dispatch problem with the additional choice of turning generators to be on or off.  In this paper, in order to improve both the exploitation and exploration abilities of the firefly algorithm (FA), a new mo...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2018